Method for increased accessibility to a human machine interface

ABSTRACT

A method is defined for providing an individual increased accessibility to a touch screen displaying first and second elements. The individual initially engages the touch screen with a contact point at a first location. The contact point is dragged across the touch screen into engagement with the first element and the first element is highlighted in response thereto. Thereafter, the individual may drag the contact point across the touch screen from the first element into engagement with the second element whereby the second element is highlighted on the touch screen and the highlight is removed from the first element. Audible announcements may accompany the contacting of the first or second elements with the contact point.

REFERENCE TO GOVERNMENT GRANT

This invention was made with government support under H133E080022awarded by the US Department of Education. The government has certainrights in the invention.

FIELD OF THE INVENTION

The invention relates generally to human-machine interfaces (HMIs), andin particular, to a device for providing improved access to thefunctions of HMIs for people with disabilities.

BACKGROUND AND SUMMARY OF THE INVENTION

The electronic touch screen is representative of current innovative HMIsfor electronic devices. With a touch screen, a user enters data bytouching virtual buttons displayed on the computer display. With a touchscreen system, the type, size, and number of the virtual buttons may bereadily changed by changing the computer's program without changes inthe associated hardware. Thus, the touch screen system offers auser-interface that may be flexibly tailored to a particularapplication. The ability to divide virtual buttons among differentscreens and to display only those screens needed by the user'sparticular task can simplify data entry. The combination of text andgraphical elements on the screen along with the virtual buttons can helpto further guide the user through the steps of data input.

Normally, a touch screen system uses a touch screen panel which isplaced directly over the viewing area of a standard computer display,for example, a cathode ray tube (“CRT”) or a liquid crystal display(“LCD”). The touch screen panel provides a signal to a computerassociated with the computer display indicating where on the surface ofthe display a stylus or finger is placed. This signal may be generated,for example, by acoustic systems detecting absorption of sound by afinger on the surface of the display; by systems using crossed beams ofinfrared light arrayed along x and y axes which are broken by thefinger; or by laminated transparent electrodes which are physicallycompressed against each other to provide a point electrical contact thatmay be/interpreted as an x and y coordinate.

Frequently, a touch screen will be used with a computer contained withina device to be used by the public. The computer may have so-calledmulti-media capabilities providing both a visual and audio programintended to help members of the public obtain information or directions.Despite the advantage of touch screen systems in such applications, likemany novel interface controls, they present a barrier to many peoplewith disabilities. Those with impaired vision perceive only thefeatureless surface of the display screen knowing that it may containone or more virtual buttons of arbitrary placement and functions. Thoseunfamiliar with the language of the legends on the buttons or those whocannot read, are also foreclosed from much of the information presentedby touch screen systems.

Therefore, it is a primary object and feature of the present inventionto provide a method for improving access to the functions of HMIs forpeople with disabilities, such as those with impaired vision; thoseunfamiliar with the language utilized by HMIs; and/or those who cannotread.

It is a further object and feature of the present invention to provide amethod for improving access to the functions of HMIs for people withdisabilities that is simple and inexpensive to implement.

It is a still further object and feature of the present invention toprovide a method for improving access to the functions of HMIs forpeople with disabilities that may be utilized without altering operationof the HMIs or the electronic devices controlled by the HMIs.

In accordance with the present invention, a method is defined forproviding an individual increased accessibility to a touch screendisplaying an element, e.g. a virtual button. The method includes thesteps of engaging the touch screen with a contact point of theindividual. The contact point is dragged across the touch screen intoengagement with the element. An audible announcement is generated thatcorresponds to the element in response to engagement of the contactpoint with the element.

It is contemplated to highlight the element on the touch screen inresponse to engagement of the element with the contact point. Thehighlighting of the element may be maintained in response todisengagement of the contact point from the touch screen. The elementmay be tapped with the contact point so as to activate the element. Inaddition, an input device having a key may be provided such that theelement may be activated by actuating the key.

The element may be a first element and the touch screen may display asecond element. The contact point may be dragged across the touch screenfrom the first element into engagement with the second element. A secondaudible announcement may be generated corresponding to the secondelement in response to engagement of the second element with the contactpoint. The first element may be highlighted on the touch screen inresponse to engagement of the first element with the contact point andthe second element may be highlighted on the touch screen in response toengagement of the second element with the contact point. It iscontemplated to remove the highlight from the first element in responseto disengagement of the contact point with the first element or inresponse to engagement of the contact point with the second element.

In accordance with a further aspect of the present invention, a methodis defined for providing an individual increased accessibility to atouch screen displaying at least one element. The method includes thestep of engaging the touch screen with a contact point of theindividual. The contact point is dragged across the touch screen intoengagement with a first element. The first element is highlighted inresponse to engagement of the contact point with the first element. Thehighlighting of the first element is maintained on the touch screen inresponse to disengagement of the contact point from the touch screen.

It is contemplated to generate an audible announcement corresponding tothe first element in response to engagement of the contact point withthe first element. The first element may be a virtual button. Thetapping of the virtual button with the contact point activates theelement. In addition, the element may be activated by actuating a key ofan input device.

The contact point may be dragged across the touch screen from the firstelement into engagement with a second element. The second element ishighlighted on the touch screen in response to engagement of the secondelement with the contact point. Thereafter, the highlight is removedfrom the first element in response to disengagement of the contact pointwith the first element or in response to engagement of the contact pointwith the second element. A second audible announcement corresponding tothe second element may be generated in response to engagement of thecontact point with the second element.

In accordance with a still further aspect of the present invention, amethod is defined for providing an individual increased accessibility toa touch screen displaying first and second elements. The individualinitially engages the touch screen with a contact point at a firstlocation. The contact point is dragged across the touch screen intoengagement with the first element and the first element is highlightedin response thereto. Thereafter, the individual may drag the contactpoint across the touch screen from the first element into engagementwith the second element whereby the second element is highlighted on thetouch screen and the highlight is removed from the first element.

The first location may be the second element and at least one of thefirst and second elements may be a virtual button. A first audibleannouncement corresponding to the first element may be generated inresponse to engagement of the contact point with the first element and asecond audible announcement corresponding to the second element may begenerated in response to engagement of the contact point with the secondelement. The highlighting of the first element on the touch screen maybe maintained in response to disengagement of the contact point from thetouch screen prior to engagement of the second element with the contactpoint.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings furnished herewith illustrate a preferred construction ofthe present invention in which the above advantages and features areclearly disclosed as well as others which will be readily understoodfrom the following description of the illustrated embodiment.

In the drawings:

FIG. 1 is a schematic view of a kiosk incorporating a keyboard system inaccordance with the present invention;

FIG. 2 is a top plan view of the keyboard system of the presentinvention;

FIG. 3 is a front plan view of an exemplary screen for display on thekiosk of FIG. 1 in an initial condition;

FIG. 4 is a front plan view of the exemplary screen of FIG. 3 inoperation;

FIG. 5 is a is a front plan view of an exemplary screen for display onthe kiosk of FIG. 1 in an alternate initial condition;

FIG. 6 is a front plan view of the exemplary screen of FIG. 5 in asecond condition; and

FIG. 7 is a front plan view of the exemplary screen of FIG. 5 in a thirdcondition.

DETAILED DESCRIPTION OF THE DRAWINGS

Referring now to FIG. 1, a schematic of a kiosk is generally designatedby a reference numeral 10. Kiosk 10 includes a conventional touch screensystem 11 incorporating touch screen 12 positioned in front of and inalignment with a standard computer display, for example, a cathode raytube or a liquid crystal display. Touch screen 12 is well known in theart and may apply a number of different technologies, including thoseemploying crossed beams of infrared light (which detect a finger orstylus by the interruption of one or more beams), and those employinglayered transparent electrodes (which detect the finger or stylus by itsdeforming layers and causing the electrodes to touch). As isconventional, touch screen 12 includes an outer, display surface 14 onwhich the stylus or finger is placed, for reasons hereinafter described.

As is conventional, touch screen 12 and the computer display areoperatively connected to port 17 and video board 21 via lines 19 a and19 b, respectively. Port 17 is, in turn, operatively connected tocentral processing unit 15 via bus 29. Touch screen 12 provides a userinterface with central processing unit 15 that is flexibly tailored to aparticular application operating in accordance with a computer softwareprogram stored on memory 25. Memory 25 also holds other data andsoftware programs including the software program that will be describedbelow forming part of the present invention. The program may beimplemented in a number ways depending on the particular device and so afunctional description of the program will be provided such as wouldallow it to be readily incorporated into standard electronic devices byone of ordinary skill in the art. Video board 21 is also connected toand receives data generated by execution of the software program bycentral processing unit 15 over bus 29 in order to provide video data totouch screen 12.

Touch screen 12 provides a high-resolution X-Y coordinate signal tocentral processing unit 15 in response to the touching or “pressing” ofthe display surface of the touch screen. Typically in operation, one ofa plurality of screens is displayed by central processing unit 15 inresponse to the computer program stored in memory 25 and/or inputs fromthe user, as heretofore described. It is intended for kiosk 10 to beaccessible to following individuals: (1) blind individuals; (2)individuals who have difficulty seeing or reading the kiosk, eitherbecause user's vision blurs the image or because the user cannotinterpret the written legends on the screen (this may be result ofproblems understanding written language or an unfamiliarity with theparticular written language or graphic element on the kiosk); (3)individuals who have difficulty hearing; and (4) individuals who havedifficulty reaching or touching the kiosk screen, for example,individuals with restricted mobility including those using wheelchairs.Further, kiosk 10 may be used use by individuals with combinations ofabove described disabilities including those who cannot hear or whocannot see the kiosk.

In order for kiosk 10 to meet the dual requirements of maintaining theadvantageous characteristics of the touch screen system 11 in presentingdata quickly to users who are not disabled and providing access to thesame information by users with disabilities with a minimum of additionaleffort, sound card 23 and keyboard 16 are provided. Sound card 23 isoperatively connected to kiosk 10 by line 27 and to central processingunit 15. Sound card 23 receives data from the bus 29 by virtue ofexecution of the computer program by central processing unit 15 toproduce sound communicated to an earphone jack or speaker 31 provided atthe front of kiosk 10.

As is conventional, keyboard 16 includes housing 18 having a circuitboard received therein. A plurality of standard hard contact keys 20project from corresponding openings 22 in housing 18. Each key 20 ofkeyboard 16 corresponds to a standard alpha-numeric character on aconventional keyboard. Small conductive plates are provided on the innerends of keys 20. When a key 20 of keyboard 16 is depressed, thecorresponding conductive plate on the inner end of the key engages thecircuit board, thereby closing a circuit on the circuit board. Inresponse, a corresponding signal associated with the depressed key 20 ofkeyboard 16 is sent to central processing unit 15 via port 17 along line19. Biasing structures are provided to urge keys 20 toward theirnon-depressed position.

Referring to FIG. 2, keyboard 16 further includes a plurality of accesskeys, hereinafter referred to collectively by the reference number 24.It can be appreciated that the orientation and locations of access keys24 on keyboard 16, as hereinafter described, may be varied withoutdeviating from the scope of the present invention. Access keys 24include up key 36 and down key 38 having generally square configurationsand being axially aligned with each other. It can be appreciated thatthe orientation and locations of the up key 36 and down key 38 may bevaried, although ideally consistency should be maintained for thebenefit of the individuals who need to use this system in a variety ofdifferent contexts. Up key 36 includes an upwardly directed chevron 40projecting from the upper surface thereof. Triangular shaped indicia 42is provided on upper surface of up key 36 at a location below anddirected towards chevron 40. Down key 38 includes a downwardly directedchevron 44 projecting from the upper surface thereof. Triangular shapedindicia 46 is provided on upper surface of down key 38 at a locationabove and directed towards chevron 44. It is intended for chevrons 40and 44 to present, by touch and sight, the shape of arrows pointingupward and downward along a generally vertical axis to identify up key36 and down key 38 to a user. As is conventional, up and down keys 36and 38, respectively, are movable between a non-depressed condition anda depressed condition.

Access keys 24 further includes back key 48 and next key 50 havinggenerally square configurations and being axially aligned with eachother. It is contemplated for back key 48 and next key 50 to bepositioned on opposites sides of down key 38. It can be appreciated thatthe orientation and locations of the back key 48 and down key 50 may bevaried, although ideally consistency should be maintained for thebenefit of the individuals who need to use this system in a variety ofdifferent contexts. Back key 48 includes a leftwardly directed chevron52 projecting from the upper surface thereof. The term “BACK” isprovided on upper surface of back key 48 at a location to the right ofchevron 52. Next key 50 includes a rightwardly directed chevron 54projecting from the upper surface thereof. The term “NEXT” is providedon upper surface of next key 38 at a location to the left of chevron 54.It is intended for chevrons 52 and 54 to present, by touch and sight,the shape of arrows pointing leftward and rightward along a generallyhorizontal axis to identify back key 48 and next key 50 to a user. As isconventional, back and next keys 48 and 50, respectively, are movablebetween a non-depressed condition and a depressed condition.

Access keys 24 further includes a help key 56 having a generally squareconfiguration and being positioned to the right of up key 36 and abovenext key 50. Diamond-shaped tactile indicator 58 (i.e., a square withone diagonal oriented vertically) projects from the upper surface ofhelp key 56. Indicia 60 may be provided on the upper surface ofdiamond-shaped tactile indicator 58 to identify help key 56 to apotential user. As is conventional, help key 56 is movable between anon-depressed condition and a depressed condition. When help key 56 ofkeyboard 16 is depressed, the corresponding conductive plate on theinner end of help key 56 engages the circuit board, thereby closing acircuit on the circuit board and generating in a corresponding signalassociated with the depressed help key 56 of keyboard 16 that is sent tocentral processing unit 15 via port 17 along line 9. Central processingunit 15 controls operation of help key 56 in accordance with thesoftware program stored on memory 25. By way of example, centralprocessing unit 15 may provide context sensitive help in the form of anaudio reading of text displayed on touch screen 12 or a textual displayon touch screen 12 in response to actuation of help key 56.

Finally, access keys 24 include action key 62 having a generallyrectangular configuration and being positioned above help key 56.Indicator 66 (i.e., an L-shaped indicia) is provided on the uppersurface of action key 62. The term “ENTER” may be provided on the uppersurface of action key 62 adjacent indicator 66 in order to identifyaction key 62 to a potential user. As is conventional, action key 62 ismovable between a non-depressed condition and a depressed condition.When action key 62 of keyboard 16 is depressed, the correspondingconductive plate on the inner end of action key 62 engages the circuitboard, thereby closing a circuit on the circuit board and generating acorresponding signal associated with the depressed help key 56 ofkeyboard that is sent to central processing unit 15 via port 17 alongline 9. Central processing unit 15 controls operation of action key 62in accordance with the software program stored on memory 25. Morespecifically, it is contemplated for action key 62 to function as astandard “enter” key on a conventional keyboard. In addition, it iscontemplated for action key 62 to be used by user to activate, mark,and/or activate elements displayed on touch screen 12 and to reread textand cause text to be spelled over speaker 31.

It is intended for kiosk 10 to present data quickly to users who are notdisabled, but to also provide access to the same information to userswith disabilities with a minimum of additional effort. Referring to FIG.3, in operation, an exemplary access screen 70 to be displayed on touchscreen 12 by central processing unit 15 in response to the internalstored program stored on memory 25, is depicted. Kiosk 10 provides auser with the ability to utilize multiple paths and methods ofcommunication. By way of example, a user may utilize touch screen 12 orkeys 20 of keyboard 16 to control operation of kiosk 10. Alternatively,it is contemplated for a disabled user to use access keys 24 incombination with touch screen 12 and/or keys 20 of keyboard 16, ashereinafter described, to control kiosk 10.

Referring to FIG. 3, in operation, is contemplated for access screen 70to display a series of elements 74 a-74 f in accordance with theinternal stored program stored on memory 25. By way of example, anelement (e.g., element 74 a) may take the form of a non-input elementcontaining information, e.g., alpha-numeric data; or an element (e.g.,elements 74 b-74 f) may take the form of a virtual button for activatingan option provided on access screen 70, for navigating through a seriesof access screens; or for a similar type of activity. It can beappreciated that elements 74 a-74 f may take alternate forms such astext, graphics, checkboxes, text/numeric entry fields or the like,without deviating from the scope of the present invention.

In order to enter a slide-to-read mode, user 82 touches access screen 70with finger 80 and drags finger 80 across access screen 70 into contactwith an element, e.g., element 74 b. By way of example, user 82 maytouch access screen 70 with finger 80 at a location 84 either inside oroutside of elements 74 a-74 f, FIG. 4, and drag finger 80 across accessscreen 70 into contact with an element, e.g., element 74 b. As a result,navigation highlight 72 will be provided about element 74 b and centralprocessing unit 15 causes an audio reading of the information providedin activation element 74 b over speaker 31. In addition, it iscontemplated for central processing unit 15 to modify the visualappearance of the information provided in activation element 74 b,(e.g., enlarge the text to facilitate the reading thereof) in responseto engagement of element 74 b with finger 80. User 82 may actuate thehighlighted element, e.g., element 74 b, by depressing action key 62with finger 80 or by tapping the highlighted element, e.g., element 74b. It is noted, however, that the simple removal of finger 80 fromengagement with the highlighted element, e.g., element 74 b, will notactuate the highlighted element. Further, it is noted that independentof the location of navigation highlight 72, in the slide-to-read mode, auser 82 may selectively actuate any actionable element, e.g., elements74 b-74 f, by tapping the such element with finger 80. Alternatively,user 82 may thereafter touch access screen 70 with finger 80 on element74 b and drag finger 80 across access screen 70 into contact with adifferent element, namely, elements 74 a or 74 c-74 f. Once again,navigation highlight 72 will be provided about the engaged element,e.g., one of elements 74 a or 74 c-74 f, and central processing unit 15will cause an audio reading of the information provided in thehighlighted element over speaker 31. It is contemplated to remove thenavigation highlight 72 from previously highlighted element 74 b inresponse to disengagement of finger 80 with the element 74 b or inresponse to engagement of finger 80 with the subsequently contactedelement, namely, elements 74 a or 74 c-74 f. Again, it is contemplatedfor central processing unit 15 to modify the visual appearance of theinformation provided in highlighted element, (e.g., enlarge the text tofacilitate the reading thereof) in response to engagement of highlightedelement with finger 80.

It can be appreciated that in certain circumstances, user 82 mayencounter highlight 72 provided about one of the elements, e.g. element74 a, FIG. 5. This may occur if user 82 enters slide-to-read mode andsubsequently removes finger 80 from element 74 a, or if user 82 invokesaccess screen 70 by actuating action key 62 (or next or back keys, 38and 48, respectively). In the event that highlight 72 is provided aboutone of the elements, e.g. element 74 a, user 82 may enter theslide-to-read mode by touching at any location on the screen, includingthe highlighted element, e.g., element 74 a, with finger 80, FIG. 6, anddragging their finger 80 across access screen 70 into contact with asecond element, e.g., element 74 b, FIG. 7. As a result, navigationhighlight 72 moves from element 74 a to element 74 b and centralprocessing unit 15 causes an audio reading of the information providedin element 74 b over speaker 31. It can be appreciated that navigationhighlight 72 may be removed from previously highlighted element 74 a inresponse to disengagement of finger 80 with the element 74 a or inresponse to engagement of finger 80 with the subsequently contactedelement, namely, elements 74 b-74 f. In addition, central processingunit 15 may modify the visual appearance of the information provided inactivation element 74 b, (e.g., enlarge the text to facilitate thereading thereof) in response to engagement of element 74 b with finger80. User 82 may actuate the highlighted element, e.g., element 74 b, byeither depressing action key 62 with finger 80 or by tapping thehighlighted element, e.g., element 74 b. It is noted, however, that thesimple removal of finger 80 from engagement with the highlightedelement, e.g., element 74 b, will not actuate the highlighted element.Further, it is noted that independent of the location of navigationhighlight 72, in the slide-to-read mode, a user 82 may selectivelyactuate any actionable element, e.g., elements 74 b-74 f, by tappingsuch element with finger 80.

If user 82 touches an element, e.g., element 74 b, with finger 80 anddrags finger 80 across access screen 70 out of and back into contactwith the same element, e.g. element 74 b, element 74 b will be actuatedupon the subsequent removal of finger 80 from such element. Thisfunctionality protects users with poor motor control from accidentlytriggering the slide-to-read function when user 82 is attempting tosimply actuate a desired element, e.g., element 74 b. However, if user82 slides into contact with any other element, e.g., elements 74 a or 74c-74 f, after sliding finger 80 off element 74 b, and then slides finger80 back into contact with element 74 b, element 74 b will not beactivated, but instead, navigation highlight 72 will be provided aboutelement 74 b and central processing unit 15 will cause an audio readingof the information provided in activation element 74 b over speaker 31.

It is understood that user 82 may also navigate access screen 70 with upkey 36 and/or down key 38. More specifically, up key 36 allows a user tomove a navigation highlight 72 upwardly on a given screen, e.g. accessscreen 70, through elements 74 a-74 f to highlight a user-desiredelement 74 a-74 f. Down key 38 allows user 82 to move navigationhighlight 72 downwardly on a given screen, e.g. access screen 70 throughelements 74 a-74 f.

It can be appreciated that the software program stored on memory 25 maygenerate a series of screens. Each screen may include one or moreelements 74 a-74 f, as heretofore described. In the present embodiment,it can be appreciated that actuation of element 74 e or of back key 48allows a user to scroll to the prior screen in the series. Similarly,actuation of next key 50 allows a user to scroll to a subsequent screenin the series. Further, it is noted that by incorporating access keys 24into a conventional keyboard and utilizing the above-describedbehaviors, accessibility to kiosk 10 for individuals with disabilitiesis improved with added convenience to users without disabilities. Theenhanced functionality of access keys 24 allows for easy navigationabout access screen 70, and through the elements 74 a-74 f on suchscreen.

Various modes of carrying out the invention are contemplated as beingwithin the scope of the following claims particularly pointing anddistinctly claiming the subject matter that is regarded as theinvention.

We claim:
 1. A method for providing an individual increasedaccessibility to a touch screen which selectively displays one of theplurality of screens thereon, a first screen of the plurality of screensincludes an element and the individual has a contact point, comprisingthe steps of: displaying the first screen on the touch screen; engagingthe touch screen with the contact point; dragging the contact pointacross the touch screen into initial engagement with the element;generating an audible announcement corresponding to the element inresponse to the initial engagement of the contact point with theelement; and selectively tapping the element with the contact pointafter initial engagement of the element with the contact point so as toactivate the element; wherein the touch screen displays a second screenof the plurality of screens in response to activation of the element. 2.The method of claim 1 comprising the additional step of highlighting theelement on the touch screen in response to engagement of the elementwith the contact point.
 3. The method of claim 2 comprising theadditional step of maintaining the highlighting of the element on thetouch screen in response to disengagement of the contact point from thetouch screen.
 4. The method of claim 1 comprising the additional stepsof: providing an input device, the input device having a key; andactuating the key to the activate the element.
 5. The method of claim 1wherein the element is a first element and the touch screen displays asecond element, comprising the additional steps: dragging the contactpoint across the touch screen from the first element into engagementwith the second element; and generating a second audible announcementcorresponding to the second element in response to engagement of thesecond element with the contact point.
 6. The method of claim 5comprising the additional steps: highlighting the first element on thetouch screen in response to engagement of the first element with thecontact point; removing the highlighting of the first element inresponse to disengagement of the contact point with the first element;and highlighting of the second element on the touch screen in responseto engagement of the second element with the contact point.
 7. Themethod of claim 5 comprising the additional steps: highlighting thefirst element on the touch screen in response to engagement of the firstelement with the contact point; removing the highlighting of the firstelement in response to engagement of the contact point with the secondelement; and highlighting of the second element on the touch screen inresponse to engagement of the second element with the contact point. 8.The method of claim 1 wherein the element is a virtual button.
 9. Themethod of claim 1 wherein removal of the contact point from engagementwith the element fails to activate the element.
 10. The method of claim1 wherein the element is a first element and the touch screen displays asecond element, comprising the additional steps: providing an inputdevice, the input device having at least one key; selectivelyhighlighting one of the first and second elements in response toactuation of the at least one key; and generating an audibleannouncement corresponding to the highlighted element.
 11. The method ofclaim 10 comprising the additional steps: dragging the contact pointacross the touch screen from the first element into engagement with asecond element; removing the highlighting the first element in responseto engagement of the contact point with the second element; andhighlighting of the second element on the touch screen in response toengagement of the second element with the contact point.
 12. The methodof claim 11 comprising the additional step of generating a secondaudible announcement corresponding to the second element in response toengagement of the contact point with the second element.